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Abstract 

Today’s college ACCUPLACER System’s Computerized Placement Test (CPT) 
plays a major role for determining whether post- secondary students are ready for 
placement into Arithmetic, Elementary Algebra, Intermediate Algebra, or College 
Mathematics courses. It is widely used in most college systems in the U.S. This 
paper discusses some of the issues to be addressed about the assessment and 
evaluation of CPT such as analyzing the reliability of the test scores and 
evaluating the validity of CPT. The results of this study suggest that the reliability 
(or correlation) between Arithmetic and Elementary Algebra test scores is less 
than Elementary Algebra and College Mathematics test scores. 
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Assessment of Computerized Placement Test for College Mathematics 
at Baltimore City Community College 
Introduction 

The ACCUPLACER System, which has been on the market since 1986 
with growing users, is designed to provide placement and advising information for 
students entering college. It includes a Computerized Placement Test (CPT) 
which is a self-paced, un-timed test given on a personal computer. This testing is 
an adaptive assessment test customized for each student. Subsequent test 
questions are based upon responses to previous questions. The first question in a 
particular test is of average difficulty, and is chosen randomly from several 
“starter” questions of the same level of difficulty. This provides a more accurate 
method of measuring an individual student’s skill levels than a general test. It also 
means that no two tests taken are precisely the same. The students will be able to 
know their result after testing session. Students’ scores will determine which 
courses they are eligible to take (College Board, 1997). 

The advantages of the CPT are the following: (a) it is the only computer 
adaptive placement test delivered over the Internet to give students course 
placement in any location; (b) it tests students with fewer questions in less time 
and with immediate results; (c) it is un-timed giving more flexibility in the 
computer lab; (d) provides access for the colleges to customize their needs such as 
faculty preferences and setting cutoff scores (Sireci, Patelis, Rizavi, Dillingham, 



& Rodriguez, 2000). 
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The purpose of this paper is two-fold: (1) to determine the reliability and 
correlation of Arithmetic with Elementary Algebra test scores and Elementary 
Algebra with College Mathematics test scores, and (2) to determine which 
mathematics exams scores are more reliable and correlated to better test students’ 
ability. The Cronbach’s Alpha and inter- item correlation have been calculated for 
Arithmetic with Elementary Algebra test scores and Elementary Algebra with 
College Mathematics test scores. Also, a comparison between the two cases has 
been made. Finally, the validity of the CPT has been reviewed and the necessary 
recommendations are suggested. 

Related Literature 

There are a number of factors that affects the reliability of any test score 
either associated with the examinees or the testing condition itself. Some of these 
are test length, test content, item difficulty, item discrimination, group 
heterogeneity, student test-wiseness, time limit, and security precautions. 
Commercial standardized tests like one used in the ACCUPLACER System are 
expected to have reliability around 0.9 (Frisbie, 1988). 

In the college, placing students in the appropriate level such as 
developmental mathematics is a difficult task particularly when it comes to cutoff 
scores. Most colleges face challenges to come up with the better cut-off scores so 
that the Accuplacer becomes the best predictor of student’s skill for the subject 
area he/she is testing. However, the College Board does not provide specific cut- 
off score for the CPT given the diversity of the students and courses across 



different colleges (Patelis, 1999). 




Assessment of Computerized Placement Test 



4 



According to BCCC (2005) Office of Institutional Research, reports for 
2002 and 2003 show that more than 50% of the students who attempted the CPT 
and were placed in Mat— 80 and Mat— 81 did not pass their assigned classes. The 
report did not indicate any particular reason why this happened. 

Since Fall 1998, BCCC mathematics department has been using the cut- 
off scores for the CPT presented in Table 1 (J. Finucci, personal communication, 
April 2, 1998). The cut-off scores were developed based on the mathematics 
proficiency statement from the College Board, comparison of cut-off scores of 
other community colleges and the expected proportion of students to pass the 
course (Leshan & Icore, 1995). 

Table 1 

Cut-off Scores for the CPT at BCCC 



Mathematics Placement 


Raw Score 


Required Courses 


Arithmetic 


Elementary Algebra 


College Math 


0-35 


<32 




Mat-80 ISP 


>35 


<32 




Mat-80 


Any score 


32-42 




Mat-80 




43-62 




Mat-8 1/BA103 




>62 


<45 


Mat-82 






45-63 


Mat-107/ Mat- 12 8/Mat- 1 1 l/Mat-125 






>63 


See Dept. Chair 







Assessment of Computerized Placement Test 



5 



According to the CPT validity study for BCCC by the College Board, the 
correlation coefficient between Arithmetic CPT scores and the grade from the 
Mat-81 course is relatively strong (0.34) as compared to the correlation between 
other CPT scores and corresponding course grade (Patelis, 1999). From this one 
may infer that the ACCUPLACER score does not reflect the placement of the 
student in the properly course level. 

Method 

The CPT is a series of Mathematics and English computerizes exams, 
which assesses students’ skills in five different areas: (1) Reading 
Comprehension, (2) Sentence Skills, (3) Arithmetic, (4) Elementary Algebra, and 
(5) College Mathematics (College Board, 1997). In order to assess the reliability 
of the test scores of the CPT Mathematic exams at BCCC, data were collected 
from August 2004 to April 2006 and results are described in detail. Statistical 
analysis using Cronbach’s Alpha and inter- item correlation was performed. 

Arithmetic exam includes 17 questions from three areas: (1) operation 
with whole numbers and fractions, (2) operations with decimals and percents, (3) 
applications and problem solving. Elementary Algebra exam has 12 questions 
from three areas: (1) operation with integers and rational numbers, (2) operations 
with algebraic expressions, (3) equations solving, inequalities, and word 
problems. College Mathematics exam has five areas with 20 questions: (1) 
algebraic operation, (2) solutions of equations and inequalities, (3) coordinate 
geometry, (4) applications and other algebra topics, (5) functions and 



trigonometry. 
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Students begin exams with one section and, depending on his/her 
performance, may advance to other sections. At BCCC, students are allowed to 
use calculators (Leshan & Icore, 1995). The report of the CPT is given in two 
scales: percentile score and total right score. Then, based on the cut-off scores of 
the college and the score of the student, placement decisions are made. The scale 
ranges from 0 to 120, with a mean of 48.29, and standard deviation of 26.21 
(Sireci et al., 2000). 

According to Sireci et al. (2000), Elementary Algebra score has estimated 
reliability of 0.93. Based on results from research done on BCCC, the CPT score 
yields with reliability around 0.90 and correlations around 0.88 (Leshan & Icore, 
1995). 

Since the CPT is computer adaptive test and students are tested on 
randomly chosen questions from the bank containing about 120 questions in each 
course areas, assessing item reliability is difficult. In this study, the reliability was 
assessed by pairing of Arithmetic with Elementary Algebra scores and 
Elementary Algebra with College Mathematics scores for the same student taking 
the CPT at same date. 

Data 

CPT scores of Arithmetic, Elementary Algebra and College Mathematics exams 
of about 4071 students who have been retested for the second time within 24 
hours were collected from a poll of all CPT scores of 10990 BCCC students who 
have taken the test from August 2004 until April 2006. The data were obtained 
from BCCC (2005, 2006) Office of Institutional Research. Then data were sorted 




Assessment of Computerized Placement Test 



7 



for Arithmetic (ARTH) with Elementary Algebra (EA) scores and Elementary 
Algebra (EA) with College Mathematics (CM) scores bases. 3082 sample data 
were obtained for EA and ARTH scores, 986 for EA and CM scores. Reliability 
and correlation values including scatter diagrams were obtained for ARTH with 
EA and EA with CM using Cronbach’s Alpha and inter- item correlations. 
Example of the data listed is given in Table 2. More sample data are included in 
the Appendix. 

Table 2 



Example of the Data Listed 



ST.ID 


subjectl 


scorel 


subject2 


score2 


1 


EA 


23 


ARTH 


26 


2 


EA 


29 


ARTH 


39 


3 


EA 


33 


ARTH 


37 



Analysis 

The data were analyzed using the SPSS (2003) software tool. For 
Elementary Algebra and Arithmetic scores (N = 3082) the results are presented in 



Table 3, Table 4, and Figure 1. 
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Table 3 



Reliability Statistics 



Cronbach’s Alpha 


Cronbach’s Alpha Based on 
Standardized Items 


N of Items 


.331 


.528 


2 



Table 4 



Summary Item Statistics 





Mean 


Minimum 


Maximum 


Range 


Variance 


N of Items 


Item Means 


30.517 


26.089 


34.944 


8.855 


39.202 


2 


Item Variances 


126.754 


21.142 


232.365 


211.223 


22307.630 


2 


Inter-Item Correlations 


.359 


.359 


.359 






2 




Figure 1. Scatter Diagram for Elementary Algebra and Arithmetic Scores 
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For Elementary Algebra and College Mathematics score (N = 986) the 
results are presented in Table 5, Table 6, and Figure 2. 

Table 5 



Reliability Statistics 



Cronbach’s Alpha 


Cronbach’s Alpha Based on 
Standardized Items 


N of Items 


.810 


.812 


2 



Table 6. Summary Item Statistics 





Mean 


Minimum 


Maximum 


Range 


Variance 


N of Items 


Item Means 


59.785 


34.178 


85.392 


51.214 


1311.437 


2 


Item Variances 


340.173 


309.144 


371.203 


62.058 


1925.620 


2 


Inter-Item Correlations 


.683 


.683 


.683 






2 




0.00 — 
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E A 



Figure 2. Scatter Diagram for Elementary Algebra and College Mathematics 



Scores 
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Discussion 

The two Cronbach’s Alpha coefficients (see Table 2 & Table 4) that are 
returned by SPSS (2003) are raw Cronbach’s Alpha and standardized Cronbach’s 
Alpha. Raw Cronbach’s Alpha is based upon item correlation. The stronger the 
items are inter-related, the more likely the test to be a consistent predictor. 
Standardized Cronbach’s Alpha is based upon item covariance. It is based on a 
linear transformation. Standardized Cronbach’s Alpha is not superior to its raw 
counterpart, but it is used when there are relatively heterogeneous variances, 
which is the case in Table 2 and Table 4. 

As shown in Table 2 to Table 4, the analysis results reported are the 
following: (a) the correlation between Arithmetic and Elementary Algebra scores 
(r = 0.359) is weaker than the correlation between Elementary Algebra and 
College Mathematics scores (r = 0.683); (b) the reliability between Elementary 
Algebra and College Mathematics scores (standardized Cronbach’s Alpha = 
0.812) is much higher than the reliability of Arithmetic and Elementary Algebra 
scores (standardized Cronbach’s Alpha = 0.528). 

The higher the Alpha the more reliable the test will be. There isn’t a 
generally agreed cut-off. Usually, Alpha = 0.7 and above is acceptable. It is a 
common misconception that if the Alpha is low, it must be a bad test. Actually, a 
test may measure several attributes/dimensions rather than one, and, thus, the 
Cronbach’s Alpha is deflated. For example, it is expected that the scores of GRE- 
Verbal, GRE-Quantitative, and GRE- Analytical may not be highly correlated 
because they evaluate different types of knowledge (Nunnally, 1994). 




Assessment of Computerized Placement Test 



11 



Therefore, analysis results of this study may imply that Elementary 
Algebra and College Mathematics tests assess student’s skill more consistently 
than Elementary Algebra and Arithmetic tests. In other words, Elementary 
Algebra test works more consistently with College Mathematics test than with 
Arithmetic. 

Conclusion and Future Research 

The study shows that the scores of the CPT administrated at BCCC for 
mathematics courses have less consistency. Some of the reasons that may 
contribute to lower consistency and need further investigations are the following: 

1. BCCC offers general mathematics reviews. It does not have any 
preparatory class for the CPT. This might contributed to lower reliability 
value as it has been seen above. Creating a preparatory class for the CPT 
taking students, informing them to take the class first before they take the 
test and making the students to be familiar with the test help to minimize 
the random error that affect the reliability result. 

2. Setting cut-off scores based on the expected proportion of students to pass 
the test will have a negative impact on the validity of the test. More 
research needs to be done in order to get more accurate cutoff scores. 

3. The number of questions given per each section in the CPT is too small to 
properly cover broad content area of each course. On average, the CPT 
tests a student a total of 17 questions from Arithmetic, Elementary 
Algebra, and College Mathematics. This does have a negative impact 
towards the validity and the reliability of the test. 
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The CPT is used only to predict academic readiness for a course. It cannot 
guarantee success in the course because other factors such as study skills, regular 
attendance, and interest towards learning can contribute greatly to success. 
Therefore, colleges are expected to have a policy for placement that includes 
these factors to be able to get a high success rate. 

The CPT is the single sole test used by BCCC to place students. Instead of 
placing student by the cut-off scale of the CPT results alone, it would be a good 
idea to use students’ academic background along with the CPT scores to place 
student in appropriate course level. This also requires further study. 
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Appendix 
Data Listed 



ST.ID 


subjectl 


scorel 


subject2 


score2 


1 


EA 


93 


CM 


26 


2 


EA 


116 


CM 


59 


3 


EA 


105 


CM 


32 


4 


EA 


86 


CM 


20 


5 


EA 


87 


CM 


30 


6 


EA 


86 


CM 


45 


7 


EA 


89 


CM 


32 


8 


EA 


71 


CM 


17 


9 


EA 


102 


CM 


20 


10 


EA 


92 


CM 


23 


11 


EA 


85 


CM 


22 


12 


EA 


89 


CM 


17 


13 


EA 


66 


CM 


29 


14 


EA 


63 


CM 


24 


15 


EA 


102 


CM 


29 


16 


EA 


88 


CM 


47 


17 


EA 


116 


CM 


59 


18 


EA 


70 


CM 


35 


19 


EA 


107 


CM 


32 


20 


EA 


68 


CM 


40 


21 


EA 


67 


CM 


27 


22 


EA 


77 


CM 


21 


23 


EA 


92 


CM 


54 


24 


EA 


102 


CM 


58 


25 


EA 


99 


CM 


56 


26 


EA 


64 


CM 


19 


27 


EA 


67 


CM 


21 


28 


EA 


111 


CM 


17 


29 


EA 


64 


CM 


33 


30 


EA 


88 


CM 


17 


31 


EA 


74 


CM 


35 


32 


EA 


97 


CM 


48 


33 


EA 


102 


CM 


48 


34 


EA 


63 


CM 


18 


35 


EA 


82 


CM 


44 


36 


EA 


112 


CM 


66 


37 


EA 


110 


CM 


23 


38 


EA 


63 


CM 


17 


39 


EA 


91 


CM 


43 


40 


EA 


71 


CM 


17 
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